-
Notifications
You must be signed in to change notification settings - Fork 953
feat(sdk-logs): Add Logger Configurator API & Trace-based + Min-Severity Filtering #5991
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat(sdk-logs): Add Logger Configurator API & Trace-based + Min-Severity Filtering #5991
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5991 +/- ##
=======================================
Coverage 95.40% 95.40%
=======================================
Files 317 319 +2
Lines 9439 9494 +55
Branches 2187 2203 +16
=======================================
+ Hits 9005 9058 +53
- Misses 434 436 +2
🚀 New features to boost your workflow:
|
pichlermarc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JacksonWeber - please also mark all features that are marked as in-development in the spec as @experimental in the TSdoc. This way we can avoid blocking https://github.com/open-telemetry/opentelemetry-js/milestone/19
Experimental features have been marked as such. Thanks for the feedback! |
pichlermarc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, overall I'm a bit confused by this feature - the spec for it seems still a bit vague and seems to duplicate functionality from existing features. 🤔 That's not this PR's issue, I feel like it's a symptom of a spec that's not quite where we'd need it to be yet.
I suppose the value proposition for having a LoggerConfigurator is to save on allocations in the pipeline - which is important for a hot path like this. (Q: @JacksonWeber is this the motivation for working on this feature?) If that's not the goal, then we could accomplish the same thing today in the LogRecordProcessor.
Though it seems to me that the optimizing LogRecordImpl to avoid expensive operations until it's properties are first accessed, and then applying filtering in the LogRecordProcessor may be a better way to go about it without adding additional concept to the public API.
The Go SIG seems to have a similar stance: open-telemetry/opentelemetry-specification#4644
| // Get logger configuration | ||
| const loggerConfig = this._sharedState.getLoggerConfig( | ||
| this.instrumentationScope | ||
| ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that emit is a hot-path, I think we should optimize here wherever possible.
One thing I noticed is that the way it's implemented right now is:
- we call getLoggerConfig()
- if it has already cached something for this Logger, it will always return that initial config
- dynamic updates are therefore not possible
Do we need the map lookup or should we just cache it in the logger? 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The primary motivation for this feature work is just to get an implementation of trace-based log sampling and minimum severity-based log sampling implemented in a way that's easy for users to enable as part of the configuration of their LoggerProvider.
Thanks for the link to the issue in the Go repo, I'm happy to refactor this PR to avoid the creation/usage of the LoggerConfigurator and just implement these two filtering strategies in the LogRecordProcessor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pichlermarc the PR has been updated to just add a new logRecordProcessor that handles both types of filtering. Please review when you have a chance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pichlermarc per our conversation in the JS SIG today I've reverted the changes here back to following the spec (creating the logger configurator).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, sorry for the churn. 🙇♂️
I think my original concern is still there. Since we don't support re-configuration we may want to cache the config in the logger instead of doing the map lookup on each logger.emit() for performance gains. Especially since we have do do a string allocation to compute the key for the map lookup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pichlermarc comments are addressed, let me know if the PR looks good now.
loggerConfiguratorFilteringLogRecordProcessor
12fb209 to
8e25840
Compare
FilteringLogRecordProcessor| /** | ||
| * Configuration for a specific logger pattern | ||
| */ | ||
| export interface LoggerPattern { | ||
| /** | ||
| * The logger name or pattern to match. | ||
| * Use '*' for wildcard matching. | ||
| */ | ||
| pattern: string; | ||
|
|
||
| /** | ||
| * The configuration to apply to matching loggers. | ||
| * Partial config is allowed; unspecified properties will use defaults. | ||
| * | ||
| * @experimental This feature is in development as per the OpenTelemetry specification. | ||
| */ | ||
| config: LoggerConfig; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we'd need an experimental annotation on all of these (the type and properties)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, updated this.
| private _getScopeKey(scope: InstrumentationScope): string { | ||
| return `${scope.name}@${scope.version || ''}:${scope.schemaUrl || ''}`; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: let's make this a utility function since we don't need to access this here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense, I've added a utils folder in the sdk-logs to house this new util function.
| // Get logger configuration | ||
| const loggerConfig = this._sharedState.getLoggerConfig( | ||
| this.instrumentationScope | ||
| ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, sorry for the churn. 🙇♂️
I think my original concern is still there. Since we don't support re-configuration we may want to cache the config in the logger instead of doing the map lookup on each logger.emit() for performance gains. Especially since we have do do a string allocation to compute the key for the map lookup.
| /** | ||
| * A LoggerConfig defines various configurable aspects of a Logger's behavior. | ||
| */ | ||
| export interface LoggerConfig { | ||
| /** | ||
| * A boolean indication of whether the logger is enabled. | ||
| * If a Logger is disabled, it behaves equivalently to a No-op Logger. | ||
| * Defaults to false (loggers are enabled by default). | ||
| */ | ||
| disabled?: boolean; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these two also need an experimental annotation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added, thanks for the catch!
Which problem is this PR solving?
This pull request introduces advanced logger configuration capabilities to the OpenTelemetry JS SDK Logs package. The main feature is the new
LoggerConfiguratorAPI, which allows users to filter logs by minimum severity, trace sampling status, and to configure loggers individually using pattern matching. These changes improve flexibility and control over log processing and exporting.Logger configuration and filtering features
LoggerConfiguratorAPI, enabling users to define per-logger configuration for severity filtering, trace-based filtering, and logger disabling, with pattern matching for logger names. (experimental/packages/sdk-logs/src/config/LoggerConfigurators.ts,experimental/packages/sdk-logs/src/types.ts,experimental/packages/sdk-logs/src/LoggerProvider.ts,experimental/packages/sdk-logs/src/internal/LoggerProviderSharedState.ts) [1] [2] [3] [4] [5] [6]Loggerclass to apply minimum severity and trace-based filtering before emitting log records, dropping logs that don't meet the configured criteria. (experimental/packages/sdk-logs/src/Logger.ts) [1] [2]Documentation and API updates
README.mdwith detailed documentation and usage examples for the new logger configuration features, including severity and trace-based filtering and per-logger configuration patterns. (experimental/packages/sdk-logs/README.md)LoggerConfig,LoggerConfigurator, andcreateLoggerConfigurator. (experimental/packages/sdk-logs/src/index.ts) [1] [2]Changelog and test updates
loggerConfiguratorfeature. (experimental/CHANGELOG.md)experimental/packages/sdk-logs/test/common/Logger.test.ts)Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist: